{
 "cells": [
  {
   "cell_type": "raw",
   "id": "a47da0d0-0927-4adb-93e6-99a434f732cf",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_position: 2\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2195672-0cab-4967-ba8a-c6544635547d",
   "metadata": {},
   "source": [
    "# Step Back Prompting\n",
    "\n",
    "Sometimes search quality and model generations can be tripped up by the specifics of a question. One way to handle this is to first generate a more abstract, \"step back\" question and to query based on both the original and step back question.\n",
    "\n",
    "For example, if we ask a question of the form \"Why does my LangGraph agent astream_events return {LONG_TRACE} instead of {DESIRED_OUTPUT}\" we will likely retrieve more relevant documents if we search with the more generic question \"How does astream_events work with a LangGraph agent\" than if we search with the specific user question.\n",
    "\n",
    "Let's take a look at how we might use step back prompting in the context of our Q&A bot over the LangChain YouTube videos."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4079b57-4369-49c9-b2ad-c809b5408d7e",
   "metadata": {},
   "source": [
    "## Setup\n",
    "#### Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e168ef5c-e54e-49a6-8552-5502854a6f01",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install -qU langchain-core langchain-openai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79d66a45-a05c-4d22-b011-b1cdbdfc8f9c",
   "metadata": {},
   "source": [
    "#### Set environment variables\n",
    "\n",
    "We'll use OpenAI in this example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "40e2979e-a818-4b96-ac25-039336f94319",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()\n",
    "\n",
    "# Optional, uncomment to trace runs with LangSmith. Sign up here: https://smith.langchain.com.\n",
    "# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
    "# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b08c52-1ce9-4d8b-a779-cbe8efde51d1",
   "metadata": {},
   "source": [
    "## Step back question generation\n",
    "\n",
    "Generating good step back questions comes down to writing a good prompt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "783c03c3-8c72-4f88-9cf4-5829ce6745d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "system = \"\"\"You are an expert at taking a specific question and extracting a more generic question that gets at \\\n",
    "the underlying principles needed to answer the specific question.\n",
    "\n",
    "You will be asked about a set of software for building LLM-powered applications called LangChain, LangGraph, LangServe, and LangSmith.\n",
    "\n",
    "LangChain is a Python framework that provides a large set of integrations that can easily be composed to build LLM applications.\n",
    "LangGraph is a Python package built on top of LangChain that makes it easy to build stateful, multi-actor LLM applications.\n",
    "LangServe is a Python package built on top of LangChain that makes it easy to deploy a LangChain application as a REST API.\n",
    "LangSmith is a platform that makes it easy to trace and test LLM applications.\n",
    "\n",
    "Given a specific user question about one or more of these products, write a more generic question that needs to be answered in order to answer the specific question. \\\n",
    "\n",
    "If you don't recognize a word or acronym to not try to rewrite it.\n",
    "\n",
    "Write concise questions.\"\"\"\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", system),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")\n",
    "llm = ChatOpenAI(model=\"gpt-3.5-turbo-0125\", temperature=0)\n",
    "step_back = prompt | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "af62af17-4f90-4dbd-a8b4-dfff51f1db95",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What are the specific methods or functions provided by LangGraph for extracting LLM calls from an event stream that includes various types of interactions and data sources?\n"
     ]
    }
   ],
   "source": [
    "question = (\n",
    "    \"I built a LangGraph agent using Gemini Pro and tools like vectorstores and duckduckgo search. \"\n",
    "    \"How do I get just the LLM calls from the event stream\"\n",
    ")\n",
    "result = step_back.invoke({\"question\": question})\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e58a714-9368-4e8e-a163-58bc4a5e56e6",
   "metadata": {},
   "source": [
    "## Returning the stepback question and the original question\n",
    "\n",
    "To increase our recall we'll likely want to retrieve documents based on both the step back question and the original question. We can easily return both like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "447ed63c-ba9f-4eaf-8ed8-b3235e45da4e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'I built a LangGraph agent using Gemini Pro and tools like vectorstores and duckduckgo search. How do I get just the LLM calls from the event stream',\n",
       " 'step_back': 'What are the specific methods or functions provided by LangGraph for extracting LLM calls from an event stream generated by an agent built using external tools like Gemini Pro, vectorstores, and DuckDuckGo search?'}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.runnables import RunnablePassthrough\n",
    "\n",
    "step_back_and_original = RunnablePassthrough.assign(step_back=step_back)\n",
    "\n",
    "step_back_and_original.invoke({\"question\": question})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a42ed79f-cb6f-490e-8186-a4a05b223857",
   "metadata": {},
   "source": [
    "## Using function-calling to get structured output\n",
    "\n",
    "If we were composing this technique with other query analysis techniques, we'd likely be using function calling to get out structured query objects. We can use function-calling for step back prompting like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "2e1fecf6-9c07-4efa-80eb-8fb15392b25f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[StepBackQuery(step_back_question='What are the steps to filter and extract specific types of calls from an event stream in a Python framework like LangGraph?')]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_core.output_parsers.openai_tools import PydanticToolsParser\n",
    "from langchain_core.pydantic_v1 import BaseModel, Field\n",
    "\n",
    "\n",
    "class StepBackQuery(BaseModel):\n",
    "    step_back_question: str = Field(\n",
    "        ...,\n",
    "        description=\"Given a specific user question about one or more of these products, write a more generic question that needs to be answered in order to answer the specific question.\",\n",
    "    )\n",
    "\n",
    "\n",
    "llm_with_tools = llm.bind_tools([StepBackQuery])\n",
    "hyde_chain = prompt | llm_with_tools | PydanticToolsParser(tools=[StepBackQuery])\n",
    "hyde_chain.invoke({\"question\": question})"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
